1. Field of the Invention
The invention relates to graphics rendering systems, and more particularly, to software tools for assisting graphics application developers.
2. Description of Related Art
Graphics rendering is the process of computing a two-dimensional image (or part of an image) from three-dimensional geometric forms. An object is considered herein to be three-dimensional if its points are specified with at least three coordinates each (whether or not the object has any thickness in all three dimensions). A renderer is a tool which performs graphics rendering operations in response to calls thereto. Some renderers are exclusively software, some are exclusively hardware, and some are implemented using a combination of both (e.g. software with hardware assist or acceleration). Renderers typically render scenes into a buffer which is subsequently output to the graphical output device, but it is possible for some renderers to write their two-dimensional output directly to the output device. A graphics rendering system (or subsystem), as used herein, refers to all of the levels of processing between an application program and a graphical output device. In many prior art systems, the graphics rendering system is coextensive with the renderer; that is, the renderer is called directly by the application program without any intervening layers of processing.
Graphics rendering systems typically feature an immediate mode interface or a retained mode interface to the application program. An immediate mode interface is a truly procedural interface in which the application program specifies each geometric primitive to the graphics rendering system every time the image is to be rendered. The rendering system does not maintain a model database from scene-to-scene, although the application program may do so. Immediate mode interfaces are highly attractive for rendering scenes where the model is changing at each frame, such as for visualization of simulations, previewing of animation sequences, or reading a series of models from a file. On the other hand, an immediate mode interface requires that the entire scene be transmitted via procedure calls to the renderer at each frame, resulting in high data bandwidth between application program and renderer. Also, the file format for a model is often simply a stream of drawing commands rather than the model itself, restricting its usefulness as a data interchange format. Immediate mode interfaces are also less conducive to providing toolkit modeling functionality to the application program, and they usually preclude a user interface toolkit that operates on objects in the scene.
In a retained mode system, sometimes called a display list system, the graphics rendering system maintains a database representation of the three dimensional model. At each frame, the rendering system traverses the retained model database and draws it. This can be instigated by a single call by the application program to the graphics rendering system, instead of a stream of drawing calls describing the entire scene. When the model changes, the application program edits or updates the model database and again asks the rendering system to render the scene. The benefits of a retained mode system include reduced bandwidth between the application program and any hardware accelerator. The file format of the model database also can be used easily as a data interchange format since it is not merely a list of procedure calls. The existence of an object database also provides an additional way of implementing a user interface toolkit and modeling functionality. Retained mode renderers can also cache rendering information and can also cache information for optimization of scene traversal. On the other hand, retained mode rendering systems have a higher overhead for editing the scene database, and they restrict application program design by forcing the scene into a system-defined data structure, usually a hierarchy, thus requiring many application programs to maintain a duplicate copy of the model in their own format.
In Mark A. Tarlton and P. Nong Tarlton, "A Framework for Dynamic Visual Applications," Proceedings of the 1992 Symposium on Interactive 3D Graphics, Cambridge, Mass., 1992, pp. 161-164, incorporated by reference herein, there is described a retained mode rendering system which implements a general purpose database system to organize the model rather than forcing the model to reside in a single system hierarchy. Such a technique attempts to provide the benefits of the retained mode system without the drawbacks. Such a high level scheme may not solve the model organization problem for all applications, however, and is not optimum for visualization applications where the scene is changing at each frame.
Conventional graphics rendering systems have a number of additional problems which limit their usefulness in one way or another. The following list describes some of the graphics rendering systems which are currently available.
GL.TM.. Silicon Graphics' GL is an immediate mode renderer used primarily for interactive graphics. GL is described in "Graphics Library Programming Guide," Silicon Graphics Computer Systems, 1991, incorporated by reference herein. It was designed as an interface to Silicon Graphics IRIS rendering hardware and does not provide a file format, hard copy output, modeling capability, or user interface tools. As its design stems from a hardware implementation, a software implementation of GL is suboptimal since caching and aggregate primitives such as polygon meshes are not supported. GL supports simple display lists which are essentially macros for a sequence of GL commands. The GL routines perform rendering operations by issuing commands to the IRIS hardware.
StarBase.TM.. Hewlett-Packard's StarBase is an immediate mode system that is very similar to GL, sharing most of its features and disadvantages. StarBase is described in "Starbase Graphics Techniques, Hewlett-Packard Company," 1991, incorporated by reference herein. Numerous device drivers are available for StarBase for outputting the rendered (i.e. two-dimensional) scene on different graphics output devices ranging from plotters to high-end 3-D graphics work stations.
RenderMan.TM.. RenderMan, by Pixar, is an immediate mode system designed primarily to support high quality rendering. RenderMan is described in Steve Upstill, "The RenderMan Companion," Addison-Wesley, Reading, Mass., 1990, incorporated by reference herein. While RenderMan does provide a simple mechanism for making display lists, no manipulation of a RenderMan display list is supported. RenderMan does not provide any support for interaction such as picking (object selection from a mouse click). As described in Tony Apodaca, "RenderMan Interface Specification Version 4.0 Beta," January, 1992, incorporated by reference herein, recent versions of the RenderMan specification provide new routines that bracket the existing RenderMan calls and allow different renderers to be used. The renderer is specified with a single call prior to rendering the scene, and it affects the entire scene. See also Pixar, "Quick RenderMan Interface and Implementation Specifications" 1992, incorporated herein by reference.
PHIGS. PHIGS is described in PHIGS Committee, A. van Dam, chair, "PHIGS Functional Description, Revision 3.0," Computer Graphics, 22(3), 1988, pp. 125-218, incorporated by reference herein, and is a descendent of GKS-3D, described in International Standards Organization, "International Standard Information Processing Systems Computer Graphics - Graphical Kernel System for Three Dimensions (GKS-3D) Functional Description," ISO Document Number 8805:1988(E), American National Standards Instituted New York, 1988, incorporated by reference herein. PHIGS was a committee-designed system for interactive 3-D graphics display. In PHIGS, the entire model database resides in a single hierarchy. Application programmers must learn a host of editing and hierarchy manipulation calls in order to effectively use the system. PHIGS employs a single renderer that supports all the rendering modes specified available in PHIGS, and does not support alternative renderers for photorealism or other effects.
PEX. PEX is an extension to the X-Windows system, defined by a serial protocol (for transmitting data between an application program and the X-Windows system) and a set of semantics which were originally derived from PHIGS. PEX has several available APIs, all of which support retained-mode, immediate-mode, and mixed-mode function calls for drawing, changing state, etc. PEX is described in "PEX Protocol Specification, Version 5.0P-X Public Review Draft," 14 Sep., 1990, Massachusetts Institute of Technology, incorporated by reference herein.
HOOPS.TM.. HOOPS, by Ithaca Software, is described in Garry Wiegand and Bob Covey, "HOOPS Reference Manual, Version 3.0," Ithaca Software, 1991, incorporated by reference herein. It is a retained mode 3-D graphics system, which organizes the model in a hierarchy whose nodes are accessed through textual strings in much the same way that files in the UNIX file system are referenced. Like PHIGS, HOOPS supports a single renderer. However, HOOPS provides more extensive scene editing functionality than PHIGS.
DORE.TM.. DORE, by Kubota, is an example of a 3-D graphics system with an object-oriented design. It is described in "Dore Programmer's Guide," Release 5.0, Kubota Pacific Computer Inc., 1991, incorporated by reference herein. DORE was designed so that scene data is renderable by many kinds of renderers, rather than a single monolithic renderer as provided by PHIGS. Renderers cannot be added dynamically to DORE, however, as the rendering methods are built into the system. In DORE, the choice of renderers is specified by setting the current rendering style in the DORE "view object". DORE then also requires the application program to attach the model to the view object before rendering. This restricts DORE to utilizing only one renderer at a time. There are other design considerations in DORE that also restrict it to using only one renderer at a time; for example, only one set of global variables is provided for maintaining the rendering state.
DORE is a retained mode system. To relieve much of the hassle associated with editing the model hierarchy and to facilitate dynamic databases and user interaction, DORE supports application callback objects, whereby an application program defines a function to be called when the callback object is encountered during scene traversal.
Inventor.TM.. Inventor is an object oriented 3-D graphics user interaction toolkit that sits on top of the GL graphics system. Like DORE, Inventor supports multiple renderers by having a renderer-specific "render" method for each object type. Inventor is a retained mode system with the entire scene residing in a "scene graph". Inventor has render action objects that take a model as a parameter. The renderer is selected by the rendering action that is used when drawing the model. The render action draws the entire model by traversing the model and calling the appropriate rendering method for each object. The usual render action is the GL rendering mode. Inventor is further complicated by including user interface widgets as objects imbedded in the scene graph. As such, implementing highly dynamic applications such as animation, scientific visualization or implicit surface modeling can be difficult. Inventor is described in Wernecke, "The Inventor Mentor", Addison-Wesley (1994), incorporated by reference herein.
Other references pertinent to the disclosure herein are the following, all incorporated by reference herein: Bergman, Fuchs, and Grant, "Image Rendering by Adaptive Refinement," Computer Graphics, 20(4), 1986, pp. 29-37; Catmull, "A Subdivision Algorithm for Computer Display of Curved Surfaces," Ph.D. Thesis, Report UTEC-CSc-74-133, Computer Science Department, University of Utah, Salt Lake City, Utah, December, 1974; Chen, Rushmeier, Miller, and Turner, "A Progressive Multi-Pass Method for Global Illumination," Computer Graphics, 25(4), 1991, pp. 165-174; Clark, "The Geometry Engine: A VLSI Geometry System for Graphics," Computer Graphics, 16(3), 1982, pp. 127-133; Foley, van Dam, Feiner, and Hughes, "Computer Graphics: Principles and Practice," Addison-Wesley, Reading, Mass., 1990; Haeberli and Akeley, "The Accumulation Buffer: Hardware Support for High-Quality Rendering," Computer Graphics, 24(4), 1990, pp. 309-318; Kelley, Winner, and Gould, "A Scalable Hardware Render Accelerator using a Modified Scanline Algorithm," Computer Graphics, 26(2), 1992, pp. 241-248; Maillot, "Three-Dimensional Homogeneous Clipping of Triangle Strips," Graphics Gems II, Academic Press, Inc., San Diego, Calif., 1991, pp. 219-231; Newell, Newell, and Sancha, "A Solution to the Hidden Surface Problem," Proceedings of the ACM National Conference, 1972, pp. 443-450; Potmesil and Hoffert, "FRAMES: Software Tools for Modeling, Rendering, and Animation of 3D Scenes," Computer Graphics, 21(4), 1987, pp. 85-93; Saito and Takahashi, "Comprehensible Rendering of 3-D Shapes," Computer Graphics, 24(4), 1990, pp. 197-206; Segal, Korobkin, van Widenfelt, Foran, and Haeberli, "Fast Shadows and Lighting Effects Using Texture Mappings," Computer Graphics, 26(2), 1992, pp. 249-252; Sillion and Puech, "A General Two-Pass Method Integrating Specular and Diffuse Reflection," Computer Graphics, 23(3), 1989, pp. 335-344; Snibbe, Herndon, Robbins, Conner, and van Dam, "Using Deformations to Explore 3D Widget Design," Computer Graphics, 26(2), 1992, pp. 351-352; Strauss and Carey, "An Object-Oriented 3D Graphics Toolkit," Computer Graphics, 26(2), 1992, pp. 341-349; Turkowski, "Design Considerations for an Object-Oriented [3D Graphics] Metafile," Proceedings of the Third Eurographics Workshop on Object-Oriented Graphics, Charnpery, Switzerland, October, 1992, pp. 163-169; Venolia, "Facile 3D Direct Manipulation," to appear in Proceedings of CHI '93, ACM/SIGCHI, Amsterdam, May, 1993; Venolia, "Automatic Alignment in Two and Three Dimensions," submitted to SIGGRAPH '93; and Wanger, "The Effect of Shadow Quality on the Perception of Spatial Relationships in Computer Generate Imagery," Proceedings of the 1992 Symposium on Interactive 3D Graphics, Cambridge, Mass., 1992, pp. 3942.
It is desirable to be able to draw a model using a variety of different kinds of renderers. For example, an application program which permits interactive editing of a three-dimensional model may benefit by using a high-speed, low-quality wire frame renderer to draw intermediate versions on a display, and by using a high-quality, low-speed Z-buffer renderer to draw the final version on a printer. However, the interchangeability of renderers is awkward at best using the rendering systems described above. Specifically, some of the above rendering systems do not support more than one renderer, while those which do, such as DORE and Inventor, bind the chosen renderer into the system configuration. Accordingly, there is a need for a graphics rendering system which provides increased flexibility to use different renderers for different purposes on the same three-dimensional model.
Another problem with the above rendering systems is that they do not support more than one renderer being active simultaneously. Simultaneous rendering of a model is desirable in a number of situations including simultaneous output of an image on both a display and a printer. As another example, it is sometimes desirable to render two different views of a model at the same time on two different parts of the same display. Simultaneous rendering would be desirable primarily in immediate mode systems, in which an application program draws an image by making a sequence of calls to the rendering system. In such systems, many application programs would be able to execute more efficiently by interspersing calls for one renderer with the sequence of calls for another renderer. Prior systems precluded such interspersed calling sequences.
Yet another problem with the above rendering systems is that those systems which supported more than one renderer, required all renderers to support at least the same geometries. For example, the Inventor rendering system could support only renderers which were able to render points, lines, and certain other predefined shapes. Simpler renderers could not be used with Inventor. More capable renderers could be used, as long as they support at least the full set of Inventor's geometric primitives, but they could not be added dynamically. The application program would have to be aware of these renderers a priori. The above rendering systems did not permit "plug-in" rendering of more or less capable renderers, with automatic detection and utilization of the renderer's features.
Yet another problem with the above rendering systems is that they do not support multipass rendering well, especially not those which support immediate- and mixed-mode rendering. A multipass renderer is one which is designed to require multiple traversals of a scene database. There are a number of reasons why a renderer might be designed for multiple passes on the scene database. For examples there may not be enough memory to render the entire frame buffer if rendering at very high resolutions. A renderer might then separate the frame buffer into strips and render each strip via a separate pass through the scene database. As another example, if the image is being rendered to multiple devices, each at a different resolution (such as when a window overlaps two monitors or when an image is being rendered to both the screen and a high resolution printer), the renderer may render to each device in a separate pass. As another example, a renderer that implements progressive refinement may make multiple passes on the scene database, continually refining the final image until the process is stopped by a user action or the image is of highest quality. Still other examples include high quality rendering algorithms involving accumulation buffers, involving using texture mapping to create shadowing and lighting effects, and involving two-pass global illumination techniques.
In a retained mode system, the application program usually does not need to know whether the renderer requires more than one pass to complete its rendering. This is because the application program makes only one call to cause the renderer to render the model, and the renderer can make as many passes as it requires before returning to the application program. In an immediate mode or mixed mode system, on the other hand, the application program plays a part in traversing the model, making multiple calls to the renderer to accomplish a single traversal. Thus for a multipass renderer, the application program would need to know how many times to traverse the model in order to complete the scene. This would require the application program to have an intimate knowledge of the renderer, and essentially would make it very difficult to support interchangeable renderers.